1. Field of the Invention
The present invention relates to an image processing apparatus, an image processing method, and a computer-readable recording medium.
2. Description of the Related Art
A conventional interpolation technology of calculating pixels that do not exist by using pixel values of an original image when extending the image is known as a technology for improving the resolution of the image. Because the interpolation technology, however, calculates pixels that do not exist from the average of multiple pixels, etc., there is a possibility that the edge with a greater luminance change is attenuated and this leads to a generally-blurred image. For this reason, recently, there is a technology of, when extending an image, improving the image resolution by extracting high-frequency components to be lost from an image different from the original image and interpolating the high-frequency components extracted from the different image into the original image. This technology is referred to as super-resolution, and example-based super-resolution from among super-resolution technologies is attracting an attention.
For example, in example-based super-resolution, a high-resolution image and a low-resolution image obtained by deteriorating the high-resolution image according to the process in which the high-resolution image is generated are in a pair. Example-based super-resolution constructs a dictionary by extracting partial areas in the same position from the paired two images. In the following descriptions, a partial area is referred to as a “patch”. Patches in any shape may be used. The following descriptions take, as an example, a case where the patch shape corresponds to rectangular areas. To construct the dictionary, the frequencies of the extracted patches are analyzed and converted into high-frequency patches and medium-frequency patches and then accumulated. The medium-frequency patches are obtained by extracting relatively high frequency bands from the low-resolution image. In example-based super-resolution, when super-resolution is applied to an input image that is a low-resolution image, a low-resolution patch is extracted from the input image, the frequencies of the extracted low-resolution patches are analyzed and then the low-resolution patch is converted into a medium frequency patch, the medium-frequency patch is collated with each pair in the constructed dictionary to select a more suitable pair, and the high-frequency patch of the selected pair is inserted at the same position in the input image that is the low-resolution image, so that the lost components are reproduced. By applying such processing to all patches, it is possible to convert the input low-resolution image into a high-resolution image.
Particularly, example-based super-resolution methods using basis sparse coding is attracting an attention because it makes it possible to obtain higher reproduction accuracy than that obtained with the above-described patch-based method. In this method, patches are not used directly, but a symbolic pattern referred to as a basis is generated from multiple patches to construct a dictionary in a scale smaller than that of the patch-based method. As in the case of the patch-based method, medium-frequency bases and the high-frequency bases have a correspondence relation. Upon super-resolution, in order to approximate the medium-frequency patches using a linear sum, a small number of medium-frequency bases and their corresponding coefficients are determined according to a coding algorithm. Such determination processing is referred to as sparse coding. Lastly, because medium-frequency bases and high-frequency bases have a correspondence relation, a high-frequency basis corresponding to the selected medium-frequency basis is multiplied by a coefficient and, using the resulting liner sum, a high-frequency patch is reproduced.
Recently, there is also a technology in which, in order to reduce the processing time, the distances each between patches of a pair of a high-resolution image and a low-resolution image obtained by deteriorating the high-resolution image are measured, the patches are classified into flat patches and non-flat patches other than the flat patches according to a set threshold, and the processing is performed on only the non-flat patches.
The above-described conventional technology however has a problem in that it is difficult to maintain a preferable relation between the reproduction accuracy and processing time. Specifically, because example-based super-resolution according to the conventional technology preforms reproduction processing per patch, there is a possibility that noise would be inserted when an appropriate high-resolution patch is not reproduced and thus the above-mentioned patch-based method requires a dictionary in which a large number of pairs of patches are accumulated. In example-based super-resolution using bases, the size of the dictionary is smaller than that of the patch-based super-resolution, but a dictionary in which a large number of pairs of bases are accumulated is still used. In the technology in which the processing is performed on only non-flat patches, an area to which super-resolution has been applied and an area to which super-resolution has not been applied are adjacent to each other at the boundary where the flat area and non-flat area are adjacent to each other, which may cause discontinuity and thus lead to visual artifacts. As a result, the conventional technology increases the processing load of searching the dictionary in which a large number of pairs of patches are accumulated to collate the patch with the pairs, and it is therefore difficult to describe that more preferable reproduction accuracy can be maintained. In other words, the reproduction accuracy and the processing time have the relation of trade-off, and it is preferable that it is possible to preferably control them according to the intended use.
In view of the above-described circumstances, there is a need to provide an image processing apparatus, an image processing method, and a computer-readable recording medium containing an image processing program that make it possible to maintain a preferable relation between the reproduction accuracy and processing time.